Tikhonov Regularization Research Articles

The Influence of YOLOv5 Hyperparameters for Construction Details Detection

Computer vision has become a fundamental area of interest in recent decades. Each area has unique data which object detection methods can analyse. However, it is important to find the most suitable parameters for the model that detects different object groups. In this research has been investigated the influence of pre-trained YOLOv5 (nano (n), small (s), medium (m), large (l), extralarge (x)) models, hyperparameters (learning rate, momentum, and weight decay) and different image augmentation (hsv_h, degrees, translate, flipud, mosaic, mixup, shear, perspective) efficiency for similar construction details detection. A newly collected dataset with twenty-two labelled categories of construction details was prepared. A total of 270 models were trained and evaluated. Every model was evaluated with 3,300 test images which backgrounds were mixed, neutral, and white backgrounds. The most accurate model was YOLOv5l with learning rate – 0.001, momentum – 0.950 and weight decay – 0.0001. This model achieved – 0.5015 (50.15%) accuracy.

Analysis of early intermediate states of the nitrogenase reaction by regularization of EPR spectra

Due to the complexity of the catalytic FeMo cofactor site in nitrogenases that mediates the reduction of molecular nitrogen to ammonium, mechanistic details of this reaction remain under debate. In this study, selenium- and sulfur-incorporated FeMo cofactors of the catalytic MoFe protein component from Azotobacter vinelandii are prepared under turnover conditions and investigated by using different EPR methods. Complex signal patterns are observed in the continuous wave EPR spectra of selenium-incorporated samples, which are analyzed by Tikhonov regularization, a method that has not yet been applied to high spin systems of transition metal cofactors, and by an already established grid-of-error approach. Both methods yield similar probability distributions that reveal the presence of at least four other species with different electronic structures in addition to the ground state E0. Two of these species were preliminary assigned to hydrogenated E2 states. In addition, advanced pulsed-EPR experiments are utilized to verify the incorporation of sulfur and selenium into the FeMo cofactor, and to assign hyperfine couplings of 33S and 77Se that directly couple to the FeMo cluster. With this analysis, we report selenium incorporation under turnover conditions as a straightforward approach to stabilize and analyze early intermediate states of the FeMo cofactor.

Enhancing genome-wide populus trait prediction through deep convolutional neural networks.

As a promising model, genome-based plant breeding has greatly promoted the improvement of agronomic traits. Traditional methods typically adopt linear regression models with clear assumptions, neither obtaining the linkage between phenotype and genotype nor providing good ideas for modification. Nonlinear models are well characterized in capturing complex nonadditive effects, filling this gap under traditional methods. Taking populus as the research object, this paper constructs a deep learning method, DCNGP, which can effectively predict the traits including 65 phenotypes. The method was trained on three datasets, and compared with other four classic models-Bayesian ridge regression (BRR), Elastic Net, support vector regression, and dualCNN. The results show that DCNGP has five typical advantages in performance: strong prediction ability on multiple experimental datasets; the incorporation of batch normalization layers and Early-Stopping technology enhancing the generalization capabilities and prediction stability on test data; learning potent features from the data and thus circumventing the tedious steps of manual production; the introduction of a Gaussian Noise layer enhancing predictive capabilities in the case of inherent uncertainties or perturbations; fewer hyperparameters aiding to reduce tuning time across datasets and improve auto-search efficiency. In this way, DCNGP shows powerful predictive ability from genotype to phenotype, which provide an important theoretical reference for building more robust populus breeding programs.

The influence of residential Environment and residential experience on psychological depression in older adults: Evidence from China and Europe

This cross-national study examined the influence of residential environment and experience on depression in older adults in China and Europe to address existing research gaps. Using data from the China Health and Retirement Longitudinal Study (CHARLS) and Survey of Health, Ageing, and Retirement in Europe (SHARE) and employing ridge regression, it was found that residential environment and experience have a significant influence on older adults' depression. The influence of residential experience aligns with the sensitivity period hypothesis. The environment in which older adults live during specific age periods leaves an imprint on their psyche. This imprint is activated when specific environments are encountered in old age, thereby influencing the level of depression. This study examined how their influence on depression contributes to understanding older adults’ housing preferences and can guide housing-related policies.

Machine Learning Models for Regional Photovoltaic Power Generation Forecasting with Limited Plant-Specific Data

Predicting electricity production from renewable energy sources, such as solar photovoltaic installations, is crucial for effective grid management and energy planning in the transition towards a sustainable future. This study proposes machine learning approaches for predicting electricity production from solar photovoltaic installations at a regional level in Italy, not using data on individual installations. Addressing the challenge of diverse data availability between pinpoint meteorological inputs and aggregated power data for entire regions, we propose leveraging meteorological data from the centroid of each Italian province within each region. Particular attention is given to the selection of the best input features, which leads to augmenting the input with 1-hour-lagged meteorological data and previous-hour power data. Several ML approaches were compared and examined, optimizing the hyperparameters through five-fold cross-validation. The hourly predictions encompass a time horizon ranging from 1 to 24 h. Among tested methods, Kernel Ridge Regression and Random Forest Regression emerge as the most effective models for our specific application. We also performed experiments to assess how frequently the models should be retrained and how frequently the hyperparameters should be optimized in order to comprise between accuracy and computational costs. Our results indicate that once trained, the model can provide accurate predictions for extended periods without frequent retraining, highlighting its long-term reliability.

Unlocking the complete blood count as a risk stratification tool for breast cancer using machine learning: a large scale retrospective study

Optimizing early breast cancer (BC) detection requires effective risk assessment tools. This retrospective study from Brazil showcases the efficacy of machine learning in discerning complex patterns within routine blood tests, presenting a globally accessible and cost-effective approach for risk evaluation. We analyzed complete blood count (CBC) tests from 396,848 women aged 40–70, who underwent breast imaging or biopsies within six months after their CBC test. Of these, 2861 (0.72%) were identified as cases: 1882 with BC confirmed by anatomopathological tests, and 979 with highly suspicious imaging (BI-RADS 5). The remaining 393,987 participants (99.28%), with BI-RADS 1 or 2 results, were classified as controls. The database was divided into modeling (including training and validation) and testing sets based on diagnostic certainty. The testing set comprised cases confirmed by anatomopathology and controls cancer-free for 4.5–6.5 years post-CBC. Our ridge regression model, incorporating neutrophil–lymphocyte ratio, red blood cells, and age, achieved an AUC of 0.64 (95% CI 0.64–0.65). We also demonstrate that these results are slightly better than those from a boosting machine learning model, LightGBM, plus having the benefit of being fully interpretable. Using the probabilistic output from this model, we divided the study population into four risk groups: high, moderate, average, and low risk, which obtained relative ratios of BC of 1.99, 1.32, 1.02, and 0.42, respectively. The aim of this stratification was to streamline prioritization, potentially improving the early detection of breast cancer, particularly in resource-limited environments. As a risk stratification tool, this model offers the potential for personalized breast cancer screening by prioritizing women based on their individual risk, thereby indicating a shift from a broad population strategy.

Retrieval algorithm of chlorophyll-a concentration for coastal waters based on ridge regression

Abstract. In this paper, we use the multiple scattering near-infrared aerosol correction model of SeaDAS to execute the atmospheric correction for Terra/Aqua MODIS remote sensing data, and also use three standard operational chlorophyll-a concentration inversion algorithms built in SeaDAS, named OC2, OC3, and OC4, to carry out chlorophyll-a concentration inversion in the Yellow Sea and East China Sea. We validate the inversion results using in-situ measured chlorophyll-a concentration data collected from the Yellow Sea and East China Sea in 2003. The results show that the inversion results of OC3 and OC4 algorithm are significantly larger than in-situ measured values, and the results of the OC2 algorithm are closer to the measured values. In view of the shortcomings of these algorithms, we proposes a chlorophyll-a concentration inversion model based on ridge regression, and carry out chlorophyll-a concentration inversion test using MODIS images covering the Yellow Sea and East China Sea 2003. The results indicate that, the new inversion model can effectively overcome the deficiencies of OC2, OC3, and OC4 algorithms. The inversion model can effectively overcome the covariance problem of OC2, OC3 and OC4 algorithms on the multivariate linear regression model, and the model passed the F-test (F=25.893, p=0.000<0.05), the mean absolute percentage error (MAPE) between the inversion values and the in-situ measured values was 21.8%, the root mean squared error (RMSE) was 0.325, and the coefficient of determination (R2) was 0.847. The accuracy and the fit degree of the new model were significantly better than those of the OC2, OC3 and OC4 algorithms. Therefore, the chlorophyll-a concentration inversion model based on ridge regression can effectively invert the chlorophyll-a concentration in the offshore.

Testing for homogeneous treatment effects in linear and nonparametric instrumental variable models

. The hypothesis of homogeneous treatment effects is central to the instrumental variables literature. This assumption signifies that treatment effects are constant across all subjects. It allows to interpret instrumental variable estimates as average treatment effects over the whole population of the study. When this assumption does not hold, the bias of instrumental variable estimators can be greater than that of naive estimators ignoring endogeneity. This article develops two tests for the assumption of homogeneous treatment effects when the treatment is endogenous and an instrumental variable is available. The tests leverage a covariable that is (jointly with the error terms) independent of a coordinate of the instrument. This covariate does not need to be exogenous. The first test assumes that the potential outcomes are linear in the regressors and is computationally simple. The second test is nonparametric and relies on Tikhonov regularization. The treatment can be either discrete or continuous. We show that the tests have asymptotically correct level and asymptotic power equal to one against a range of alternatives. Simulations demonstrate that the proposed tests attain excellent finite sample performances. The methodology is also applied to the evaluation of returns to schooling and demand estimation in a fish market.

Variational data assimilation with finite-element discretization for second-order parabolic interface equation

Abstract In this paper, we propose and analyze a finite-element method of variational data assimilation for a second-order parabolic interface equation on a two-dimensional bounded domain. The Tikhonov regularization plays a key role in translating the data assimilation problem into an optimization problem. Then the existence, uniqueness and stability are analyzed for the solution of the optimization problem. We utilize the finite-element method for spatial discretization and backward Euler method for the temporal discretization. Then based on the Lagrange multiplier idea, we derive the optimality systems for both the continuous and the discrete data assimilation problems for the second-order parabolic interface equation. The convergence and the optimal error estimate are proved with the recovery of Galerkin orthogonality. Moreover, three iterative methods, which decouple the optimality system and significantly save computational cost, are developed to solve the discrete time evolution optimality system. Finally, numerical results are provided to validate the proposed method.

Iterative reconstruction methods and the resolution principle for fast-ion loss detector measurements

Abstract Fast-ion loss detectors (FILDs) are crucial for analyzing fast-ion dynamics in magnetically confined fusion plasmas. A core challenge is to derive an accurate ion velocity distribution, requiring treatment of thousands of remapped camera frames for a full discharge. The ill-posed nature of this task necessitates regularization with a well-chosen regularization parameter and computationally efficient methods. In this work, we introduce the `resolution principle,' a novel criterion for selecting the optimal regularization parameter, providing a distinction between genuine features and artefacts smaller than the diagnostic resolution in the reconstruction, thereby preventing misinterpretations. This principle, coupled with three iterative reconstruction techniques--Kaczmarz's method, coordinate descent, and Cimmino's method--demonstrates enhanced reconstruction capabilities compared to conventional methods like Tikhonov regularization. Utilizing these techniques allows rapid processing of measurements from full discharges, removing the computational bottleneck and facilitating between-discharge reconstructions. By reconstructing 6000 camera frames from an ELMy H-mode discharge at ASDEX Upgrade, we capture the temporal evolution of gyroradii and pitch angles, unveiling a direct correlation between pitch-angle behaviour and changes in the toroidal magnetic field for a specific subset of lost ions accelerated by edge-localized modes (ELMs) to energies approximately twice that of the injection energy.

Tomographic reconstructions of the fast-ion phase space using imaging neutral particle analyser measurements

In this paper we demonstrate how the inversion, in energy and major radius (E, R) coordinates, of imaging neutral particle analyser (INPA) measurements can be used to obtain the fast-ion distribution. The INPA is most sensitive to passing ions with energies in the range (20–150) keV and pitches near 0.5 in the core and 0.7 near the plasma edge. Inversion of synthetic signals, via 0th-order Tikhonov and Elastic Net regularization, were performed to demonstrate the capability of recovering the ground truth fast-ion 2D phase-space distribution resolved in major radius and energy, even in the presence of moderate noise levels (10%). Finally, we apply our method to measure the 2D phase-space distribution in an MHD quiescent plasma at ASDEX Upgrade and find good agreement with the slowing down fast-ion distribution predicted by TRANSP.

Effective combinations of features in predicting the range of incident ions using machine learning

Machine learning techniques were used to predict the range of ions in material, even when stopping power data is unknown. Simple features such as atomic and mass numbers were used in learning process. Two different machine learning models, Decision Tree (DT) and Kernel Ridge Regression (KRR), were used to assess each feature’s importance and the effect of combining them. The results showed that the chosen features were sufficient for accurate prediction. Furthermore, it was found that the combination and features are essential for accurate results. Analysis by KRR established a relationship between the range of the implanted ions and the energy of the incident ions, showing that the range is approximately proportional to the energy in the dataset used in this work.

Through their eyes: Multi-subject brain decoding with simple alignment techniques

Abstract To-date, brain decoding literature has focused on single-subject studies, that is, reconstructing stimuli presented to a subject under fMRI acquisition from the fMRI activity of the same subject. The objective of this study is to introduce a generalization technique that enables the decoding of a subject’s brain based on fMRI activity of another subject, that is, cross-subject brain decoding. To this end, we also explore cross-subject data alignment techniques. Data alignment is the attempt to register different subjects in a common anatomical or functional space for further and more general analysis. We utilized the Natural Scenes Dataset, a comprehensive 7T fMRI experiment focused on vision of natural images. The dataset contains fMRI data from multiple subjects exposed to 9,841 images, where 982 images have been viewed by all subjects. Our method involved training a decoding model on one subject’s data, aligning new data from other subjects to this space, and testing the decoding on the second subject based on information aligned to the first subject. We also compared different techniques for fMRI data alignment, specifically ridge regression, hyper alignment, and anatomical alignment. We found that cross-subject brain decoding is possible, even with a small subset of the dataset, specifically, using the common data, which are around 10% of the total data, namely 982 images, with performances in decoding comparable to the ones achieved by single-subject decoding. Cross-subject decoding is still feasible using half or a quarter of this number of images with slightly lower performances. Ridge regression emerged as the best method for functional alignment in fine-grained information decoding, outperforming all other techniques. By aligning multiple subjects, we achieved high-quality brain decoding and a potential reduction in scan time by 90%. This substantial decrease in scan time could open up unprecedented opportunities for more efficient experiment execution and further advancements in the field, which commonly requires prohibitive (20 hours) scan time per subject.

Interaction of black carbon surface mass with meteorological variables and spatial pattern across the 36 states of tropical Nigeria

Black carbon (BC) has been linked to cardio-pulmonary diseases and lung cancer. It is also the second largest contributor to climate change, after carbon dioxide (CO2), Effective mitigation of BC is of much benefits, but it requires detailed knowledge of its spatial distribution, as well as the influence of local meteorology on the spread of the pollutant. Many countries in sub-Sahara Africa including Nigeria, lack data on BC level. There is also limited information on meteorological impact on BC spatial distribution. This study therefore assessed the spatial and temporal distribution of BC across the 36 states of Nigeria and the influence of meteorology on the pollutant spread. Data on BC surface mass (BCMASS) and seven meteorological variables (air temperature, relative humidity, rainfall, wind speed, planetary boundary layer height (PBLH), atmospheric pressure and wind direction) were obtained from Modern Era Retrospective analysis for Research (MERRA-2). Spatial distribution of BCSMASS across 36 states of Nigeria, and interaction with meteorology was studied using correlation analyses, Bayesian ridge regression with marginal maximum likelihood estimation, and a Hybrid Single-Particle Lagrangian Integrated Trajectory (HYSPLIT) model. There was significant (p < 0.05) variation in BCSMASS across the states. The meteorological variables, except wind direction, significantly influenced BCSMASS and they accounted for 63% of the variation. Five-day backward trajectory modelled for each month of the year revealed that northern Nigeria is majorly under the influence of the dry and dusty tropical continental wind, and this could partially account for the lower BCSMASS concentration in the region, compared to southern Nigeria. Based on population-weighted exposure, states in southern Nigeria, especially the south-eastern part, should be prioritized in planning mitigation, for effective climate action.

In situ build surface topography determination in electron beam powder bed fusion

AbstractElectron optical imaging is the most promising process monitoring method in electron beam powder bed fusion. State of the art in modern machines is the installation of a single detector in the top center of the build chamber. Exemplary applications are the reconstruction of digital twins of manufactured parts to compare their dimensional accuracy or analysing the top surface of each layer to identify surface features like pores or material transport. Multi-detector systems are currently under research and have shown great potential in reconstructing the surface topography in situ. A recently developed ray tracing model, describing the image formation process, allows to formulate design guide lines for multi-detector systems and provides a method for the computation of the normal vector field of the build surface. This work utilizes the recent progress and presents a newly developed four-detector system and an updated computation chain, which enable build surface topography reconstruction in situ in every layer of a build process. The computation chain contains a normal integration algorithm, which employs Tikhonov regularization to cope with measurement irregularities. The integration method is validated with ex situ measured as-built surfaces. Additionally, first applications are demonstrated and connections to process parameter changes illustrated.

Short-term industrial load forecasting based on error correction and hybrid ensemble learning

Accurate industrial load forecasting is a prerequisite for ensuring the smooth operation of the power system. Due to the strong fluctuation and complex characteristics of industrial loads, it is difficult to accurately predict short-term power demand. To address this issue, this paper proposes a deep learning prediction model based on hybrid ensemble and error correction. The proposed model is divided into two phases: in the first phase, deep power features are extracted from multivariate data through a hybrid ensemble strategy consisting of Random Subspace, Boosting, Ensemble Pruning, and Multi-Objective Molecular Dynamics Theory Optimization Algorithm (MMDTOA). First, the strategy splits high-dimensional industrial data into multiple sub-datasets. Subsequently, for the features of each sub-dataset, the proposed MMDTOA is applied to perturb the parameters of GRU to generate base learning machines that balance accuracy and diversity. Finally, these base learning machines are integrated by kernel ridge regression stacking. Among them, the two-stage selection strategy and co-evolutionary strategy are embedded into the MMDTOA to enhance the optimization searching effect; in the second stage, a combined error correction strategy is proposed by utilizing the residual information in the prediction results. By combining the dynamic Gaussian error correction function and GRU error correction model, the prediction accuracy of the ensemble model is further improved; Experimental results on real Korean datasets show that the proposed method achieves a minimum value of 3.684% and 7.266% in NMAE and NRMSE, which outperforms eight comparative models, such as SVM, ELM, and CNN, with higher accuracy and robustness.

Regularization of vertical derivatives of potential field data using Morozov's discrepancy principle

AbstractThe calculation of the vertical derivatives of potential field methods can be carried out in a stable manner by Tikhonov regularization, but this procedure requires the appropriate selection of a regularization parameter. For this purpose, we introduce a criterion based on Morozov's discrepancy principle that uses a preliminary approximation given by the vertical derivative of the smoothed data. The smoothing may be performed by a physical or a mathematically based low‐pass filter. The filtered data are computed only for estimating the regularization parameter; once it is found, we evaluate the regularized vertical derivative from the original data (not from the smoothed one) in the frequency domain. We verified from experiments with noise‐corrupted synthetic data, as well as gravity and magnetic field data, that the regularized vertical derivative has about the same smoothness as the one obtained from filtered data, but true anomalies are more easily distinguished from noise and the shapes of the anomalies are better preserved.

An evaluation of variable selection methods using Southern Africa solar irradiation data

Dimensionality poses a challenge in developing quality predictive models. Often when modelling solar irradiance (SI), many covariates are considered. Training such data has several disadvantages. This study sought to identify the best variable embedded selection method for different location and time horizon combinations from Southern Africa solar irradiance data. It introduced new variable selection methods into solar irradiation studies, namely penalised quantile regression (PQR), regularised random forests (RRF), and quantile regression forest (QRF). Stability analysis, performance and accuracy metric evaluations were used to compare them with the common lasso, elastic and ridge regression methods. The QRF model performed best in all locations followed by the shrinkage methods on hourly data. However, it was found that QRF is not sensitive to associations through correlations, thereby ignoring the relevance of variables while focusing on importance. Among the shrinkage methods, the lasso performed best in only one location. On the 24-hour horizon, elastic net dominated the performances among the shrinkage methods, but QRF was best in three locations of the six considered. Results confirmed that variable selection methods performed differently on different situational data sets. Depending on the strengths of the methods, results were combined to identify the most paramount variables. Day, total rainfall, and wind direction were superfluous features in all situations. The study concluded that shrinkage methods are best in cases of extreme multicollinearity, while QRF is best on data sets with outliers or/and heavy tails.

Reservoir temperature prediction based on characterization of water chemistry data—case study of western Anatolia, Turkey

Reservoir temperature estimation is crucial for geothermal studies, but traditional methods are complex and uncertain. To address this, we collected 83 sets of water chemistry and reservoir temperature data and applied four machine learning algorithms. These models considered various input factors and underwent data preprocessing steps like null value imputation, normalization, and Pearson coefficient calculation. Cross-validation addressed data volume issues, and performance metrics were used for model evaluation. The results revealed that our machine learning models outperformed traditional fluid geothermometers. All machine learning models surpassed traditional methods. The XGBoost model, based on the F-3 combination, demonstrated the best prediction accuracy with an R2 of 0.9732, while the Bayesian ridge regression model using the F-4 combination had the lowest performance with an R2 of 0.8302. This study highlights the potential of machine learning for accurate reservoir temperature prediction, offering geothermal professionals a reliable tool for model selection and advancing our understanding of geothermal resources.

A Technique for SAR Significant Wave Height Retrieval Using Azimuthal Cut-Off Wavelength Based on Machine Learning

A Technique for SAR Significant Wave Height Retrieval Using Azimuthal Cut-Off Wavelength Based on Machine Learning